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ABSTRACT 

The  dynamic  nature  of  littoral  regions  requires  a  reconnaissance  approach  that  can  rapidly  quantify  environmental 
conditions.  Inadequate  estimation  of  these  conditions  can  have  substantial  impacts  on  the  performance  of  Naval  systems. 
Given  that  expeditionary  warfare  operations  can  occur  over  timescales  on  the  order  of  hours,  exploitation  of  video 
imagery  from  tactical  vehicles  such  as  Unmanned  Aircraft  Systems  (UAS)  has  proved  to  be  a  reliable  and  adaptive 
solution.  Tactical  littoral  products  that  can  be  created  by  exploiting  UAS  imagery  include  estimates  of  surf  conditions, 
dominant  wave  period,  wave  direction,  nearshore  currents,  and  bathymetry.  These  vehicles  can  fly  for  durations  of  1-2 
hours  at  altitudes  of  less  than  1 000  m  (beneath  typical  cloud  cover)  to  obtain  imagery  at  pixel  resolutions  better  than  1 
m.  The  main  advantage  of  using  imaging  sensors  carried  by  these  vehicles  is  that  the  data  is  available  in  the  region  of 
operational  interest  where  other  data  collection  approaches  would  be  difficult  or  impossible  to  employ.  The  through-the- 
sensor  exploitation  technique  we  have  developed  operates  in  two  phases.  The  first  step  is  to  align  individual  image 
frames  to  a  common  reference  and  then  georegister  the  alignment  into  a  mapped  image  sequence.  The  second  phase 
involves  signal  processing  of  pixel  intensity  time  series  (virtual  sensors)  to  determine  spatial  relationships  over  time. 
Geophysical  relationships,  such  as  linear  wave  dispersion,  can  then  be  applied  to  these  processed  data  to  invert  for 
environmental  parameters  such  as  bathymetry. 
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1  INTRODUCTION 


1.1  Background 

The  remote  determination  of  environmental  parameters  in  the  littoral  zone  has  been  an  interest  of  both  scientific  and 
military  communities  for  decades.  A  major  factor  in  this  determination  is  that  the  littoral  environment  is  dynamic,  having 
variability  over  many  temporal  and  spatial  scales,  and  it  is  dangerous  in  that  in-situ  sampling  using  instrumentation 
carries  significant  risk  to  the  equipment  and  operational  personnel  responsible  for  deployment.  Optical  remote  sensing 
provides  a  preferred  solution  to  rapidly  and  accurately  sample  shallow  water  and  surf  zone  conditions  that  have  visible 
signatures.  Figure  1,  a  video  image  of  a  littoral  region,  shows  such  signatures  with  identifiable  parameters  including 
wave  propagation,  surf,  longshore  currents,  shoreline  locations,  and  bathymetry.  Aside  from  direct  determination  of 
conditions  (littoral  reconnaissance)  to  be  used  in  mission  planning,  these  same  parameters  can  often  serve  as  forcing  or 
boundary  conditions  within  numerical  models  to  allow  forecasting  of  additional  environment  parameters,  possibly  into 
the  future  as  long  as  conditions  are  unchanged. 
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Figure  1.  Littoral  snapshot  image  from  Santa  Rosa  Island,  FL  obtained  from  an  altitude  of  approximately  100  m 
above  mean  sea  level.  Intensity  patterns  associated  with  identifiable  wave  crests  allow  the  determination  of 
bathymetry,  surf  zone  breaking  patterns,  and  the  shoreline. 


Of  particular  interest  is  the  exploitation  of  environmental  sensing  that  can  be  easily  configured  for  use  with  manned  and 
unmanned  airborne  platforms.  This  concept  extends  back  to  operational  requirements  in  World  War  II  that  lead  to  the 
development  of  algorithms  to  determine  beach  gradients  from  airborne  photographs  [1].  These  algorithms  were 
partitioned  into  two  similar  approaches.  The  first  approach,  known  as  the  wave  velocity  method,  utilizes  photographs  of 
wave  patterns  in  combination  with  a  geophysical  relationship  known  as  the  linear  dispersion  relation  to  deduce  water 
depths.  This  relation  (in  the  absence  of  cross-shore  directed  mean  currents)  is  commonly  expressed  in  terms  of  wave 
celerity  or  phase  speed  c  as: 
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where  X  =  wave  length,  T  =  wave  period,  and  h  =  depth  of  water.  Inversion  of  this  equation  yields  depths  in  the  direction 
of  wave  propagation  for  wave  celerity  values  remotely  sensed.  In  shallow  water,  where  the  depth  is  approximately  less 
than  10%  of  the  wave  length,  the  inversion  equation  reduces  to  h  —c2 / g.  The  wave  velocity  approach  used  two  images 
collected  over  a  known  time  interval  to  estimate  the  wave  lengths  and  phase  speeds  used  as  inputs  to  equation  (1).  In  the 
second  approach,  the  wave  period  method,  only  a  single  image  was  required,  where  changes  in  measured  wave  lengths 
across  the  image  were  related  to  water  depths  under  an  assumption  of  narrow-banded  swell  wave  propagation.  The 
procedure  was  to  measure  wave  lengths  as  far  seaward  as  possible.  These  deep-water  wave  lengths  were  then  used  to 
determine  T  following  the  dispersion  limit  of  X  =  gT2 / 2n .  Knowing  X  and  T  allowed  solution  for  h  in  equation  (1). 

Both  methods  suffered  from  problems  relating  to  wave  crest  determination,  image  scaling,  timing  inaccuracies  and  errors 
in  geodetic  mapping,  which  resulted  in  useful  applications  being  limited  to  only  a  handful  of  occasions  [1].  In  fact  in  the 
same  article  (p.  91),  Rear  Admiral  Wyatt,  the  Hydrographer  to  the  Navy,  is  quoted  as  stating  that  “such  a  lot  of  flying 
has  to  be  done  to  obtain  photographs  under  just  the  right  conditions  that  it  would  be  a  most  expensive  means  of  survey”. 

1.2  Recent  Advances 


Advances  in  littoral  characterization  using  airborne  remote  sensing  came  with  introduction  of  Fourier  methods  to 
estimate  frequency  and  wave  length  from  multiple  image  frames.  This  so  called  time  series  imagery  (or  more  recently 
full  motion  video  or  “FMV”)  was  a  significant  advantage  over  the  WWII  approaches  in  that  the  error  associated  with 
estimating  shoaling  wave  frequency  and  wave  length  was  greatly  reduced.  In  addition,  a  more  statistically  robust 
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solution  could  be  found  by  examining  multiple  wave  periods  as  is  common  in  a  complex  sea.  Although  the  ability  to 
estimate  wave  properties  using  this  spectral  approach  was  indicated  as  early  as  the  1960s  (see  [2]),  extensive  use  of  the 
dispersion  method  to  estimate  bathymetry  from  time  series  imagery  did  not  occur  until  the  late  1990s  [3,  4]  when 
imaging  equipment  and  mapping  procedures  had  improved.  Although  mostly  limited  to  stationary  platform  such  as 
towers,  some  littoral  observations  were  obtained  from  manned  aircraft  [5,6]. 

One  important  outcome  of  these  methods  relating  to  expected  operational  accuracy  comes  from  the  paper  by  [7]  that 
used  detailed  pressure  gauge  observations  to  demonstrate,  that  outside  of  the  surf  zone,  the  linear  dispersion  relation  is 
highly  accurate,  with  average  depth  estimation  errors  on  the  order  of  6%  of  the  observed  depth.  Corroboration  of  these 
expectations  using  remotely  sensed  FMV  data  was  provided  by  [8]  where  time  series  imagery  from  aircraft  having  a 
dwell  of  2  minutes  exhibited  accuracies  within  approximately  95%  of  in  situ  truth.  These  similar  findings  indicate  that 
Fourier  methods  of  analyzing  time  series  imagery  are  applicable  for  use  in  the  bathymetry  inversion  problem.  This 
approach  (described  next  in  section  2.1),  serves  as  the  basis  for  our  littoral  environmental  reconnaissance  products  that 
can  be  derived  from  tactical  imagery  provided  from  unmanned  aircraft  systems. 

2  METHODS 


2.1  Spectral  analysis  of  time  series  imagery 

Although  a  number  of  approaches  to  analyzing  littoral  time  series  imagery  have  been  proposed  (see  [9]  for  a  summary), 
most  follow  a  procedure  that  is  generalized  here.  Given  a  time  sequence  of  images,  pixel  intensities  can  be  mapped  to 
spatial  dimensions  I(x,y,t)  using  established  photogrammetric  transformations  [10].  Typically  the  separation  time 
between  images  is  constant  allowing  straightforward  application  of  spectral  processing  within  the  frequency  domain,/ 
This  is  accomplished  by  computing  a  three-dimensional  normalized  Fourier  Transform  to  allow  the  determination  of 
cross-spectral  coefficients  Cy  over  a  number  of  pre-defmed  frequency  bands  and  sensor  positions  (ij).  These  coefficients 
effectively  quantify  the  spatial  phase  relationships  within  the  image  in  that  intensity  variations  are  assumed  proportional 
to  wave  surface  slopes,  therefore  tracking  of  coherent  intensity  variations  is  equivalent  to  imaging  waves  (height 
observations  are  not  required  to  use  the  dispersion  relation).  A  cross-spectral  model  of  the  wave  field  is  then  constructed 
as  a  spatially-temporally  coherent  superposition  of  traveling  sinusoids  as  a  function  of  wavenumber,  k : 

Cy  =  exp((/fAAv;/  +  ky  Ay.. ))  (2) 

where  k  =  2nlX  and  Ax  and  Ay  are  the  separation  distances  between  sensors.  A  weighted  cost  function  can  be  established 
as  w  |c  -c.  |  (with  w,  the  spectral  coherence)  such  that  a  nonlinear  minimization  can  be  applied  to  determine  the  best- fit 

k  over  multiple  frequencies.  These  values  are  finally  passed  to  the  linear  dispersion  relation  to  estimate  water  depth,  d,  in 
which  the  inverted  water  depth  best  matches  the  propagation  of  all  resolved  gravity  waves  that  follow  the  dispersion 
relationship.  In  our  implementation,  confidence  intervals  are  also  computed  using  the  quality  of  the  wavenumber 
minimization  solution. 

A  number  of  pre-conditioning  steps  are  typically  performed  on  I(x,y,t )  such  as  spatial  trend  normalization  and  temporal 
demeaning  to  minimize  inter-frame  gain  fluctuations.  An  important  constraint  is  the  size  of  the  analysis  patch  size  used 
in  the  minimization  as  larger  patches  will  lead  to  more  robust  estimates,  however,  since  the  bottom  depth  is  typically 
regarded  as  constant  within  the  analysis  patch,  fewer  independent  depth  estimates.  An  initial  guess  of  the  deep-water 
wave  direction  relative  to  the  Cartesian  coordinate  system  is  commonly  provided  and  a  frequency  range  of  interest  is 
selected  to  exclude  temporal  variability  outside  of  what  might  include  the  dominant  well  peak.  Typically  wave  lengths 
provided  as  seeds  to  the  minimization  algorithm  are  constrained  within  the  algorithm  to  be  within  the  deep  water  and 
shallow  water  (2-m)  limits  of  the  dispersion  equation. 

2.2  Application  to  Unmanned  Aircraft  Systems 

The  Fourier  analysis  described  above  assumes  that  the  imagery  can  be  mapped  accurately.  For  a  stationary  platform 
(such  as  a  camera  mounted  on  a  tower)  or  for  an  aircraft  with  sophisticated  positioning  and  inertial  measurement 
hardware  such  as  AROSS  described  by  [11],  this  transformation  can  be  accomplished  effectively.  Use  of  an  Unmanned 
Aircraft  System  (UAS)  for  this  purpose  is  an  entirely  different  matter.  Our  goals  were  to  work  with  UAS  platforms 
commonly  available  operationally  which  includes  the  Tier  I  and  Tier  II  classes  of  small,  limited  endurance  UAS.  With 
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wingspans  of  less  than  3  m  and  total  weights  on  the  order  of  5  kg  (see  Table  1),  these  SUAS  systems  were  never 
designed  to  carry  the  advanced  GPS  or  inertial  measurement  hardware  used  on  manned  aircraft  that  can  often  cost  more 
than  the  aircraft  system  itself.  While  camera  payload  orientation  and  positional  values  are  available  in  video- 
synchronized  metadata  streams,  investigations  have  shown  these  data  are  insufficient  to  allow  direct  estimation  of 
camera  pointing  angles  for  frame-by-frame  georectification  at  required  accuracies  [12].  In  practice,  compass  errors  in 
azimuth  of  over  1 0  degrees  are  common  and  payload- to-payload  variations  are  neglected  through  the  use  of  a  single  set 
of  calibration  parameters. 

Table  1.  Small  Unmanned  Aircraft  System  (SUAS)  Technical  Specifications 


SPECIFICATION  S* 

Performance  capabilities: 


Raven  RQ11B 

Physical  dimensions: _ 

Wingspan  1.4  m  (4.5  ft) 

Body  length  0.9  m  (3  ft) 

Weight  1.9  kg  (4.2  lbs) 

Deployment  procedures: 

Hand  launched;  deep-stall  landing. 


Range/Speed  10  km  /  32-81  km/h 
Endurance  60- 110  min 

Payload  Forward/side  look  EO/IR 
Maximum  Altitude  4267  m  ( 14,000 ft) 

*(from  www.avinc.com) 


Because  of  these  inaccuracies,  we  chose  a  somewhat  dramatic  deviation  from  traditional  photogrammetric  mapping 
procedures.  Rather  than  mapping  each  individual  frame  to  the  ground  plane  using  the  supplied  metadata  (Figure  2,  left), 
we  implemented  a  frame  alignment  procedure  that  works  with  the  video  frames  to  provide  relative  matching 
transformations  between  any  frame  in  the  clip  sequence  and  a  user  selected  reference  frame.  The  procedure  uses  the 
scale-invariant  feature  transform  or  SIFT  algorithm  suggested  by  [13]  to  provide  the  feature  matching  used  in  the 
alignment  (Figure  2,  right).  The  entire  alignment  is  then  mapped  according  to  an  optimized  solution  of  the  metadata 
associated  with  the  matched  features  in  the  clip  frames.  This  procedure,  when  applied  to  stable  flight  paths,  provides  a 
robust  solution  to  the  frame-to-frame  inaccuracies. 


Figure  2.  Traditional  (left)  versus  revised  (right)  image  mapping  procedures.  Raw  image  frame  time  series  are 
designated  as  the  stacks  of  squares.  Given  the  inaccurate  metadata  associated  with  Small  Unmanned  Aircraft 
Systems,  the  revised  procedure  produced  more  robust  image  time  series  and  image  mosaics  (rounded  rectangles). 


In  addition  to  the  modified  mapping  procedure,  SUAS-based  imagery  collection  presents  additional  challenges  to  littoral 
reconnaissance.  For  example,  the  large  number  of  operational  considerations  relating  to  an  unmanned  aircraft  flight  in 
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terms  of  collection  strategies  and  sensor  performance  is  quite  large.  A  simulation  tool  was  developed  to  estimate  the 
quality  of  bathymetric  surfaces  produced  for  a  variety  of  remote  sensing  scenarios.  This  tool  is  based  upon  both 
theoretical  and  empirical  relations  [3,  7,  8]  between  wavenumber  estimation  accuracy  and  motion  imagery  collection 
parameter  choices.  The  simulation  variables  involved  include  platform  flight  altitude,  aircraft  speed,  grazing  angle,  lens 
horizontal  field  of  view,  number  of  horizontal  and  vertical  pixels,  number  of  desired  bathymetry  estimates,  typical  local 
water  depth  and  typical  local  wavelength.  An  important  use  of  these  simulations,  which  corroborates  previous  findings, 
is  that  accuracy  of  the  inversion  method  is  largely  dependent  upon  the  image  dwell  and  size  of  the  spatial  analysis 
region.  An  example  is  shown  for  a  set  of  specific  flight  parameters  in  Figure  3.  For  this  simulation,  dwell  over  target  is 
the  critical  parameter  in  that  the  estimated  depth  errors  increase  exponentially  as  the  dwell  decreases.  We  determined 
that  the  shortest  useful  dwell  was  on  the  order  of  50  seconds  to  obtain  depths  with  less  than  20%  error.  Suitable  spatial 
resolution  and  temporal  sampling  rates  were  more  easily  obtained  using  the  SUAS  hardware  with  pixel  resolution  and 
sample  rate  requirements  being  <  3  m  and  >  2  Hz  respectively. 


Figure  3.  Optical  Bathymetry  Simulation  for  a  SUAS  flying  at  an  altitude  of  1500  m.  Simulation  shows 
bathymetric  errors  for  wide  field-of-view  lens  (20°)  on  gimbaled  turret.  Spatial  coverage  of  imaged  area  is 
approximately  128  x  128  m.  Larger  field-of-view  (or  higher  altitudes)  will  increase  coverage,  but  eventually  will 
decrease  imaging  resolution  below  what  is  required  to  observe  wave  crests. 


3  RESULTS 


3.1  Surrogate  data  results 

Given  the  difficulty  in  obtaining  flight  permissions  for  unmanned  aircraft  operations  in  littoral  regions,  we  leveraged 
existing  video  data  of  a  littoral  region  obtained  from  a  long-term  camera  deployment  at  Santa  Rosa  Island,  FL.  This 
station,  part  of  the  Argus  network  described  by  [14],  provides  high  quality  digital  time  series  images  that  can  be  used  to 
test  the  bathymetric  inversion  methods  described  in  Section  2.1.  In  addition,  high-resolution  bathymetric  surveys  were 
completed  on  a  quarterly  basis  using  jetski-based  equipment  similar  to  that  described  by  [15]. 

Approximately  250  video  image  sequences  were  selected  for  analysis  under  the  criterion  of  having  visible  waves  and 
being  within  +/-  two  weeks  of  a  bathymetric  survey.  Each  of  these  sequences,  originally  10  minutes  in  duration,  were 
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shortened  to  50  seconds  to  be  consistent  with  that  expected  for  a  SUAS  flight  data  collection.  Average  computed  wave 
directions  were  typically  from  176  degrees  relative  to  N,  matching  the  onshore  propagation  direction  perpendicular  to 
the  measured  shoreline.  Dominant  wave  periods  ranged  from  5.5  to  12.5  seconds  and  typical  wave  lengths  at  the 
offshore  edge  of  the  domain  were  on  the  order  of  61  m.  Depths  in  the  imaged  region  varied  between  2  and  12  m. 

When  compared  to  the  surveyed  bathymetry  being  used  as  ground  truth,  bathymetric  errors  were  on  the  order  of  13.5% 
of  the  measured  depth.  An  example  comparison  is  shown  in  Figure  4.  While  this  estimate  is  slightly  larger  than  the 
theoretical  capability  under  optimal  conditions  (-6%),  it  is  most  likely  representative  of  what  can  be  obtained  from  full 
motion  video  provided  by  SUAS  platforms  being  flown  under  operational  scenarios.  The  average  bias  of  these  estimated 
surfaces  from  ground  truth  was  42  cm.  In  the  calculation  of  these  statistics,  depths  with  estimated  errors  (based  on  the 
wavenumber  minimization)  were  excluded. 


Figure  4.  Simulated  bathymetric  inversion  of  UAS  imagery  provided  by  tower  camera  surrogate  for  validation  of 
analytic  approaches.  Data  from  29  January  2009,  1300  GMT. 


3.2  SUAS  data  results 

An  experiment  was  conducted  26-30  January  2009  at  Santa  Rosa  Island,  FL  to  test  tactical  environmental  reconnaissance 
approaches,  including  remote  bathymetry  estimation  using  SUAS.  The  operational  area  extended  approximately  4  km  in 
the  alongshore  where  bathymetric  groundtruth  was  obtained  using  a  variety  of  acoustic  surveying  methods.  Actual  data 
SUAS  collection  opportunities  to  validate  the  mapping  and  inversion  approaches  described  above  were  limited  to  time 
periods  where  meteorological  conditions  were  within  flight  thresholds  (no  rain,  winds  less  than  20  knots,  etc.)  and 
oceanographic  conditions  showed  visible  wave  signatures.  Flight  concepts  of  operations  (or  “conops”)  were  developed 
to  obtain  imagery  that  could  be  used  in  a  moasicing  process  to  identify  shorelines,  sand  bars  and  surf  zones.  An  example 
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mosaic,  created  using  the  alignment  process  diagramed  in  Figure  2,  is  shown  in  Figure  5.  From  the  figure,  shoreline 
locations  and  even  complex  sand  bar  configurations  can  be  determined  visually.  Algorithms  have  been  developed  to 
enable  the  automated  extraction  of  these  features  by  creating  time-averaged  and  variance  exposures  where  intensity 
changes  such  as  between  water  and  land  are  obvious. 


Figure  5.  Geo-rectified  mosaic  from  SUAS  imagery  obtained  during  overflight  of  Santa  Rosa  Island,  FL  on  27 
January  2009. 


The  conops  for  the  collection  of  similar  image  sequences  to  be  analyzed  for  bathymetry  were  slightly  different  in  that  the 
flights  were  conducted  starting  over  land  centered  on  the  shoreline  to  maximize  look  angle  into  the  wave  fronts  and  to 
maximize  dwell.  Unfortunately,  out  of  the  10  sorties  totaling  approximately  8  hours  flight  time,  visible  wave  patterns 
during  this  low  energy  period  were  insufficient  to  derive  bathymetry.  Maximum  dwells  over  the  surf  zone  were  limited 
to  less  than  40  seconds.  Follow-on  exercises  using  alternate  platforms  are  planned. 

4  DISCUSSION 

Over  the  past  several  years,  there  has  been  substantial  research  and  development  effort  placed  on  the  use  of  Unmanned 
Aircraft  Systems  to  collect  oceanographic  information  supporting  littoral  environmental  reconnaissance.  Significant 
progress  has  been  made  with  respect  to  assessment  of  sensor  quality  and  data  analysis  procedures.  A  robust  procedure 
for  creating  mapped  image  sequences  has  been  developed  that  uses  image-to-image  alignments  prior  to  geo-rectification 
from  optimized  metadata.  These  procedures  have  allowed  the  rapid  creation  of  impressive  reconnaissance  products  such 
as  mapped  time-averaged  mosaics.  Shorelines,  sand  bars,  and  surf  zones  can  be  directly  digitized  from  these  type 
products.  However,  operational  implementation  of  validated  algorithms  to  derive  wave  angles,  wave  periods,  wave 
lengths  and  bathymetry  has  been  handicapped  by  platform  instability  that  limits  continuously  dwell  over  a  region  to  well 
less  than  what  is  required  based  on  theoretical  and  empirical  simulations.  Based  on  the  surrogate  data  analysis,  our 
expectation  is  that  a  new  class  of  airframe  with  a  gimbaled  camera  will  allow  longer  dwells  of  sufficient  length  to  enable 
operational  inversion  of  bathymetry  using  SUAS. 
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